Training Algorithm for Restricted Boltzmann Machines Using Auxiliary Function Approach
نویسندگان
چکیده
あらまし 深層学習の重要な一要素として,レイヤーワイズの pre-trainingがある.レイヤーワイズの pre-trainingの 一つとして制約つきBoltzmannマシン (RBM)が有名である.RBMにはBernoulli-Bernoulli型とGaussian-Bernoulli 型があり,従来法の学習アルゴリズムとしてContrastive Divergence法が有名である.本発表ではBernoulli-Bernoulli 型,Gaussian-Bernoulli型の両方の最尤学習アルゴリズムと,最適化規準として新たに最大再構築確率を導入し,そ の学習アルゴリズムに焦点を当て,経験的に高速で安定に収束する補助関数法による新たな更新アルゴリズムの導出 を行う.そして,人工データによる収束性能の比較実験を行い,その挙動に対して議論する. キーワード 深層学習,制約付き Boltzmannマシン,補助関数法,最大再構築確率学習
منابع مشابه
Conditional Restricted Boltzmann Machines for Multi-label Learning with Incomplete Labels
Standard multi-label learning methods assume fully labeled training data. This assumption however is impractical in many application domains where labels are difficult to collect and missing labels are prevalent. In this paper, we develop a novel conditional restricted Boltzmann machine model to address multi-label learning with incomplete labels. It uses a restricted Boltzmann machine to captu...
متن کاملTraining Restricted Boltzmann Machine via the Thouless-Anderson-Palmer free energy
Restricted Boltzmann machines are undirected neural networks which have been shown to be effective in many applications, including serving as initializations for training deep multi-layer neural networks. One of the main reasons for their success is the existence of efficient and practical stochastic algorithms, such as contrastive divergence, for unsupervised training. We propose an alternativ...
متن کاملQuantum Inspired Training for Boltzmann Machines
We present an efficient classical algorithm for training deep Boltzmann machines (DBMs) that uses rejection sampling in concert with variational approximations to estimate the gradients of the training objective function. Our algorithm is inspired by a recent quantum algorithm for training DBMs [1]. We obtain rigorous bounds on the errors in the approximate gradients; in turn, we find that choo...
متن کاملA Hybrid Algorithm based on Deep Learning and Restricted Boltzmann Machine for Car Semantic Segmentation from Unmanned Aerial Vehicles (UAVs)-based Thermal Infrared Images
Nowadays, ground vehicle monitoring (GVM) is one of the areas of application in the intelligent traffic control system using image processing methods. In this context, the use of unmanned aerial vehicles based on thermal infrared (UAV-TIR) images is one of the optimal options for GVM due to the suitable spatial resolution, cost-effective and low volume of images. The methods that have been prop...
متن کاملInductive Principles for Learning Restricted Boltzmann Machines (DRAFT: August 25, 2010)
We explore the training and usage of the Restricted Boltzmann Machine for unsupervised feature extraction. We investigate the many different aspects involved in their training, and by applying the concept of iterate averaging we show that it is possible to greatly improve on state of the art algorithms. We also derive estimators based on the principles of pseudo-likelihood, ratio matching, and ...
متن کامل